Reinforcement learning (RL) has shown promise in robotics, but deploying RL on real vehicles remains challenging due to the complexity of vehicle dynamics and the mismatch between simulation and reality. Factors such as tire characteristics, road surface conditions, aerodynamic disturbances, and vehicle load make it infeasible to model real-world dynamics accurately, which hinders direct transfer of RL agents trained in simulation. In this paper, we present a framework that decouples motion planning from vehicle control through a spatial and temporal alignment strategy between a virtual vehicle and the real system. An RL agent is first trained in simulation using a kinematic bicycle model to output continuous control actions. Its behavior is then distilled into a trajectory-predicting agent that generates finite-horizon ego-vehicle trajectories, enabling synchronization between virtual and real vehicles. At deployment, a Stanley controller governs lateral dynamics, while longitudinal alignment is maintained through adaptive update mechanisms that compensate for deviations between virtual and real trajectories. We validate our approach on a real vehicle and demonstrate that the proposed alignment strategy enables robust zero-shot transfer of RL-based motion planning from simulation to reality, successfully decoupling high-level trajectory generation from low-level vehicle control.

Traditional methods for developing and evaluating autonomous driving functions, such as model-in-the-loop (MIL) and hardware-in-the-loop (HIL) simulations, heavily depend on the accuracy of simulated vehicle models and human factors, especially for vulnerable road user safety systems. Continuation of development during public road deployment forces other road users including vulnerable ones to involuntarily participate in the development process, leading to safety risks, inefficiencies, and a decline in public trust. To address these deficiencies, the Vehicle-in-Virtual-Environment (VVE) method was proposed as a safer, more efficient, and cost-effective solution for developing and testing connected and autonomous driving technologies by operating the real vehicle and multiple other actors like vulnerable road users in different test areas while being immersed within the same highly realistic virtual environment. This VVE approach synchronizes real-world vehicle and vulnerable road user motion within the same virtual scenario, enabling the safe and realistic testing of various traffic situations in a safe and repeatable manner. In this paper, we propose a new testing pipeline that sequentially integrates MIL, HIL, and VVE methods to comprehensively develop and evaluate autonomous driving functions. The effectiveness of this testing pipeline will be demonstrated using an autonomous driving path-tracking algorithm with local deep reinforcement learning modification for vulnerable road user collision avoidance.

Decision-making for automated driving remains a challenging task. For their integration into real platforms, these algorithms must guarantee passenger safety and comfort while ensuring interpretability and an appropriate computational time. To model and solve this decision-making problem, we have developed a novel approach called COR-MP (Conservation of Resources model for Maneuver Planning). This model is based on the Conservation of Resources theory, a psychological concept applied to human behavior. COR-MP is based on various driving parameters, such as comfort, safety, or energy, and provides in real-time a profit value that enables us to quantify the impact of a decision on the decision-maker. Our method has been tested and validated through closed-loop simulations using RTMaps middleware, and preliminary results have been obtained by testing COR-MP on a real vehicle.

The current approach for new Advanced Driver Assistance System (ADAS) and Connected and Automated Driving (CAD) function development involves a significant amount of public road testing which is inefficient due to the number miles that need to be driven for rare and extreme events to take place, thereby being very costly also, and unsafe as the rest of the road users become involuntary test subjects. A new development, evaluation and demonstration method for safe, efficient, and repeatable development, demonstration and evaluation of ADAS and CAD functions called VehicleInVirtualEnvironment (VVE) was recently introduced as a solution to this problem. The vehicle is operated in a large, empty, and flat area during VVE while its localization and perception sensor data is fed from the virtual environment with other traffic and rare and extreme events being generated as needed. The virtual environment can be easily configured and modified to construct different testing scenarios on demand. This paper focuses on the VVE approach and introduces the coordinate transformations needed to sync pose (location and orientation) in the virtual and physical worlds and handling of localization and perception sensor data using the highly realistic 3D simulation model of a recent autonomous shuttle deployment site in Columbus, Ohio as the virtual world. As a further example that uses multiple actors, the use of VVE for VehicleToVRU communication based Vulnerable Road User (VRU) safety is presented in the paper using VVE experiments and real pedestrian(s) in a safe and repeatable manner. VVE experiments are used to demonstrate the efficacy of the method.

The short-loading cycle is a repetitive task performed in high quantities, making it a great alternative for automation. In the short-loading cycle, an expert operator navigates towards a pile, fills the bucket with material, navigates to a dump truck, and dumps the material into the tipping body. The operator has to balance the productivity goal while minimising the fuel usage, to maximise the overall efficiency of the cycle. In addition, difficult interactions, such as the tyre-to-surface interaction further complicate the cycle. These types of hard-to-model interactions that can be difficult to address with rule-based systems, together with the efficiency requirements, motivate us to examine the potential of data-driven approaches. In this paper, the possibility of teaching an agent through reinforcement learning to approach a dump truck's tipping body and get in position to dump material in the tipping body is examined. The agent is trained in a 3D simulated environment to perform a simplified navigation task. The trained agent is directly transferred to a real vehicle, to perform the same task, with no additional training. The results indicate that the agent can successfully learn to navigate towards the dump truck with a limited amount of control signals in simulation and when transferred to a real vehicle, exhibits the correct behaviour.

Automated driving systems are an integral part of the automotive industry. Tools such as Robot Operating System and simulators support their development. However, in the end, the developers must test their algorithms on a real vehicle. To better observe the difference between reality and simulation--the reality gap--digital twin technology offers real-time communication between the real vehicle and its model. We present low fidelity digital twin generator and describe situations where automatic generation is preferable to high fidelity simulation. We validated our approach of generating a virtual environment with a vehicle model by replaying the data recorded from the real vehicle.

Autonomous vehicle (AV) algorithms need to be tested extensively in order to make sure the vehicle and the passengers will be safe while using it after the implementation. Testing these algorithms in real world create another important safety critical point. Real world testing is also subjected to limitations such as logistic limitations to carry or drive the vehicle to a certain location. For this purpose, hardware in the loop (HIL) simulations as well as virtual environments such as CARLA and LG SVL are used widely. This paper discusses a method that combines the real vehicle with the virtual world, called vehicle in virtual environment (VVE). This method projects the vehicle location and heading into a virtual world for desired testing, and transfers back the information from sensors in the virtual world to the vehicle. As a result, while vehicle is moving in the real world, it simultaneously moves in the virtual world and obtains the situational awareness via multiple virtual sensors. This would allow testing in a safe environment with the real vehicle while providing some additional benefits on vehicle dynamics fidelity, logistics limitations and passenger experience testing. The paper also demonstrates an example case study where path following and the virtual sensors are utilized to test a radar based stopping algorithm.

Parallel driving is a novel framework to synthesize vehicle intelligence and transport automation. This article aims to define digital quadruplets in parallel driving. In the cyber-physical-social systems (CPSS), based on the ACP method, the names of the digital quadruplets are first given, which are descriptive, predictive, prescriptive and real vehicles. The objectives of the three virtual digital vehicles are interacting, guiding, simulating and improving with the real vehicles. Then, the three virtual components of the digital quadruplets are introduced in detail and their applications are also illustrated. Finally, the real vehicles in the parallel driving system and the research process of the digital quadruplets are depicted. The presented digital quadruplets in parallel driving are expected to make the future connected automated driving safety, efficiently and synergistically.

Digital quadruplets aiming to improve road safety, traffic efficiency, and driving cooperation for future connected automated vehicles are proposed with the enlightenment of ACP based parallel driving. The ACP method denotes Artificial societies, Computational experiments, and Parallel execution modules for cyber-physical-social systems. Four agents are designed in the framework of digital quadruplets: descriptive vehicles, predictive vehicles, prescriptive vehicles, and real vehicles. The three virtual vehicles (descriptive, predictive, and prescriptive) dynamically interact with the real one in order to enhance the safety and performance of the real vehicle. The details of the three virtual vehicles in the digital quadruplets are described. Then, the interactions between the virtual and real vehicles are presented. The experimental results of the digital quadruplets demonstrate the effectiveness of the proposed framework.